Implementing Quake live matchmaking

Intro

We used to play warsow at work every day. And while warsow ffa is fun and everything, I still much more prefer quake live duel. The only problem is absence of any kind of matchmaking or "play now" button. It was present back when quake live was free to play. But then eventually was removed and got plenty of complains from players that it was just teleporting them to random server. Quake live community and developers are already made huge effort to make easy all kind of services creation around quake live. As a kind of exercise I decided to implement that play now button for duels, with good skills balancing. Using Scala, Play framework and Akka.

Chicken and egg problem

The project was doomed for failure right from the start. It is important to know how to make estimations, that's why all that questions on the interview about the amount of gas stations in London. Lets do simple math

11% is quite huge number of auditory to convince - people already have their preferred way to matchmake - like irc or discord. This is our best case scenario - in worst case we will get way more people needed to use our service for matchmaking to work. In case if we want wait only 1 minute instead of 10 - then we need to convince 113% of target group.

Chicken and egg problem solution - on demand server spawn

After I have almost implemented matchmaking I decided to make the whole thing usable a bit more. At least for me. So I have added server on demand spawn. Which works like this - you need a server - you click create server and get a server. Either public or private server. With public server I have also decided to add some kind of matchmaking - server title displays your rating and name - so people can find you through regular server browsers.

User identification

We need to identify users. The nice way that also goes through other systems is steamId. Quake live dedicated server will report its events with steamId of user even is related to. qlstats.net have an api to query user statistics as json providing steamId. Steam openId allows us to login user by redirecting it to the steam website. After login was done we get steamId. Read more about steam open id at Steam WEB Api documentation.

Game launching

Ok, lets say we have matched 2 players. How can we kick the into the game? Steam provides Steam browser protocol which allows us to generate a link like steam://connect/95.85.4.47:27960/AXCQW Here is 95.85.4.47:27960 - ip and port. And AXCQW is password. When user clicks on a link like these steam launcher automatically launches a game and connects him to the quake server.

Design

I decided to split the whole system into next separate components/microservices

Matchmaking

Matchmaking is done using external service to provide statistics (qlstats.net). The statistics comes in form of Glicko rating. Then we just found 2 users with almost the same statistics.

Happy path overview

Lets take a look at happy path to understand how this parts work together

ui

Simple javascript with some react. Displays basic information. Matchmakes via websocket to the front. Sends on demand server requests via simple http post.

front

Front was written in scala/akka/play and is intended to process http requests from user and serve static content. It also makes requests to 3rd-paty websites like qlstats.net and steam openid login. Interesting part is matchmaking that requires long wait - was implemented over websocket.

queue

Every minute goes through list of users and if good match found (glickos are the same +-200) then it asks instanceMaster to create a server for matched users. When server is created it reports back to front that sends via websocket to ui of matched users that match was found.

instanceSlave

Spawns and monitors quake live dedicated servers. Can have more than one quake live dedicated server. Registers within instanceMaster on the start - reporting its capacity (how many quake live dedicated servers it can run).

Monitoring of quake live dedicated servers is done via zmq. I have not found good library for scala that will support zmq authentication, so I have made a small wrapper around jeromq. We tell that user for whom we created server either on demand request or via matchmaking is an owner. In case of matchmaking there are 2 owners. If owner who requested server has not joined the server within 10 minutes - we kill the server.

instanceMaster

Registers instanceSlaves capacity and current usage. When request for server creation comes it routes it to random instanceSlave.

Various interesting bugs

Operator== type checking absence

== accepts Any as its argument. Which mean no compiler error will be generated if you do this val x:String = "1" val y:Int = 1 val z:Boolean = x == y Use cats/scalaz === to avoid this problem. Also declare type from time to time - at least on what function returns :) This problem striked me when I compared 2 sets using ==. One set was containing user ids, the other one was containing userInfo structures.

pattern matching not type checked

Well, it is actually type checked. But when you do this on Any - it is obviously not. So when you receive something from other actor and do match {case ... case ...} it will be hard to catch during the refactoring. The solution is not to send tuples directly, but use separate case classes for messages going between actors. In this way compiler will at least check that you matching what you sending.

Actor postStop is not called

One of the most time consuming bugs occurred when I decided that making a blocking call in actors receive would be a good idea. After sending PoisonPill to the actor - it has not been stopped - postStop was not called. All kind of other weird side effects occurred. So simply do not block actors ExecutionContext forever.

Lessons learned

Conclusion

Go visit it at http://hurtmeplenty.space. On demand server spawn is feature I am using a lot. Now you can just send a link to a friend who you want to play with.